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COMPinER-4MPLEMENTED EDUCATIONAL PRODUCT RECOMMENDATION SYSTEM 



BACKGROUND 

This invention relates to a computer implemented product recommendation 

system. 

A number of current Internet-based systems provide customers with the ability 
to browse an inventory of available products, such as books or toys, using a Web 
browser applications (e.g., Netscape Navigator or Microsoft Internet Explorer 
designed for the Worldwide Web) which executes on the customer's personal 
computer and which communicates over the Internet with a Web server application at 
centralized server computer. Many of these systems provide the ability to search for 
and select particular products, for example, by supplying keywords that are present in 
a product description. Some systems also allow the customer to specify a particular 
category of products, for example products appropriate to a particular age group, 
products in a particular price range, types of toys (e.g., dolls, action figures, games, 
etc.), or products from a particular manufacturer or publisher. A customer can then 
select and purchase a product, for example, by providing a credit card number to 
charge for the product, and an address to which the product is to be shipped. 

SUMMARY 

In a general aspect, this invention addresses some of the shortcomings of 
many currently available product search and selection approaches. In particular, the 
invention provides a way of recommending products to a customer by matching 
information characterizing an individual for whom the products are intended to 
information characterizing each of the products in a product inventory. The invention 
is particularly direct to recommendation of educational products based on test or 
survey results related to a child. 
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In one aspect, in general, the invention is a computer-implemented system for 
recommending educational products, such as books, toys, games, puzzles, crafts and 
software. The system includes a storage for data characterizing relationships between 
product attributes within a set product attributes, and a storage for data associating 
each of a number of products with one or more of the product attributes. The set of 
product attributes includes a set of educational skills. The system also includes 
software stored on a computer-readable medium for causing a computer to perform 
the functions of determining one or more desired product attributes, accessing 
attribute relationship data from the storage for data characterizing relations between 
the product attributes, determining related product attributes based on the desired 
product attributes and the attribute relationship data, accessing product association 
data from the storage for data associating products with product attributes, and 
determining a matching set of the products using the product association data and the 
desired and related product attributes. 

The invention can include one or more of the following features: 

The storage for data associating each of the products with one or more of the 
product attributes includes a database table for holding a number of records. Each 
record associates one of the products with one of the product attributes. 

The relationships between product attributes include inclusion of particular of 
the educational skills within some category of educational skills. Determining related 
product attributes then includes determining educational skills that are in the same 
category of educational skills as a desired educational skill. The relationships 
between product attributes can further include inclusion of a number of categories of 
educational skills within a broader category of educational skills. 

Detennining one or more desired educational skills includes providing a 
questionnaire including a plurality of questions and accepting responses to the 
questions. 

Detennining an equivalent grade level associated with each of the determined 
educational skills based on the accepted responses. 
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Determining one or more desired educational skills includes accepting test 
scores obtained on a standardized test and determining an equivalent grade level 
associated with each of the one or more educational skills based on the accepted test 
scores. 

Determining an equivalent grade level includes comparing the accepted test 
scores with statistical data characterizing a distribution of test scores for a grade level. 

Determining one or more desired product attributes includes determining test 
scores for an individual, and mapping the tests scores to a quantity related to one of 
the educational skills, such as an equivalent grade level associated with the skill. For 
instance, the system can directly administer a set of questions and grade answers from 
the individual to determine test scores, or the system can accept test scores for a test 
administered outside the control of the system, such as a standardized test 
administered to school children. 

The invention includes one or more of the following advantages: 
Use of a common set of product attributes, such as educational skills, for both 
characterizing products, and for representing desired properties for products provides 
an efficient mechanism for matching desired attributes to products. 

Products can be matched to vaiying numbers of attributes, thereby providing a 
mechanism for representing products that are focused on a single attribute, as well as 
those spanning a number of attributes. 

Mapping test scores for an individual into one or more educational skills and 
associated grade equivalents provides a mechanism for using the results of a variety 
of tests without having to modify the data that associates the products with 
educational skills. In this way, particularly relevant educational products can be 
recommended to children based on their scores on a diagnostic test administered by 
the system, as well an on their score on standardized tests, such as those administered 
by a school. 

Other features and advantages of the invention are apparent from the 
following description, and from the claims. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

FIG.l is a block diagram of a client/server software system, which is used to 
recommend products to a user of the system; 

FIG. 2 is a block diagram of data stored and maintained on the server 
computer; 

FIG. 3 is a diagram of database tables and interrelationships of the tables; 
FIGS. 4A-D are a tabular representation of a skill hierarchy stored in the skill 
hierarchy table; 

FIG. 5 is a diagram of a many-to-many skill/product mapping; 

FIGS. 6A-C are diagrams which illustrate approaches for obtaining 
information about a child; 

FIG. 6A is a data flow diagram that illustrates direct input from a user, and use 
of a learning style survey to determine learning styles of the child; 

FIG. 6B is a data flow diagram that illustrates use of a skills test to determine 
a skills assessment for the child; 

FIG. 6C is a data flow diagram that illustrates use of scores obtained from a 
standardized test to determine a skills assessment for the child; 

FIG. 7 is a sample set of learning style questions; 

FIGS. 8A-E is a sample skills test; and 

FIG. 9 is a diagram that illustrates the relationship of skills and selected 
products. 

DESCRIPTION 

Referring to FIG. 1, a number of client computers 110 and a server computer 
140 are coupled over communication links 128 and 132, respectively, to Internet 130. 
Together, software that executes on a client computer 110 and on server computer 140 
forms a client/server software system. A user 120 uses this software system to obtain 
a list of recommended products for a child 122 based on inputs related to that child 
that are provided by user 120, and optionally to select and purchase some of the 
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recommended products. The system determines characteristics of child 122 from the 
inputs provided by user 120 and matches those characteristics to particular products. 
In particular, the system uses the input provided by user 120 to determine learning 
styles or skills that are related to the child, and matches those learning styles or skills 
to an inventory of educational products. The system sorts the matched products 
according to a variety of properties of the products and recommends the matched 
product to the user according to their sorted order, for example by displaying a sorted 
list of the product names and related graphics for the top matching products. In 
embodiments that include the function of on-line purchasing, user 120 can select and 
purchase one or more of the recommended products. 

Although the description of this embodiment focuses on a system for 
providing recommendations for educational products for a child, alternative 
embodiments provide recommended products for other classes of individuals, 
including adults, and for other types of products or items that are not necessarily 
commercial products. In general, user 120 is not necessarily the same person as child 
122, for instance user 120 may be the child's parent. However, the user and the child 
can indeed be the same individual. 

In some operating modes, the system uses test scores 1 14 to determined 
characteristics of child 122. In one such operating mode, a testing application 116 
executes on client computer 110 using executable software provided on a CD-ROM 
118. Child 122 responds to questions generated by testing application 116, and the 
responses are used to determine scores for individual or groups of questions. In 
another such operating mode, child 122 has previously taken a standardized test 124, 
and user 120 provides test scores 114 based on the graded standardized test 

In some embodiments, a third-party computer 160 is also coupled to Internet 
130 over a communication link 164. Client computer 110 communicates with third 
party computer 160, which then communicates with server computer 140, in addition 
to or instead of client computer 110 communicating directly with server computer 
140. In one such embodiment, third party computer 160 collects some inputs related 
to child 122, and forwards data derived from those inputs to server computer 140. 
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Server computer 140 then either communicates directly with client computer 110, or 
indirectly though third party computer 160, to provide product recommendations to 
user 120. 

A server-based system of the type described above has been deployed and is in* 
commercial use. A server computer 140 is accessible to client computers 1 10 at the 
URL (uniform resource locator) http://www.smarterkids.com/. The content of the 
Web pages that are publicly accessible at that site are incorporated herein by 
reference. A third-party computer 160 that communicates both with a client computer 
1 10 and a server computer 140 has been deployed and is accessible at the URL 
http://www.wehelpkids.com/ . The content of the Web pages that are publicly 
accessible at that site are also incorporated herein by reference. 

Server computer 140 includes a database 150 that includes information related 
to an inventory of products. A database server application 148 is coupled to database 
150 and provides an interface to the information stored in database 150 to other 
software modules that execute on server computer 140. In this embodiment, database 
150 is a relational database, which includes a number of interrelated tables. Database 
server application 148 is an SQL server that accepts queries according to an SQL 
language syntax and provides responses to those queries. Database server application 
148 has the capabilities of executing stored procedures. Each stored procedure 
implements a relatively complex query that may involve multiple separate accesses to 
database 150. These stored procedures are stored on server computer 140 in database 
procedures 146. Database procedures 146 are stored on a file system, such as on a 
magnetic disk storage, on which the procedures a stored in individual files in a source 
(SQL) or compiled form. 

Server computer 140 also executes a Web server application 142. In some 
embodiments, the functions performed by server computer 140 are split among 
several server computers, for example, having Web server application 142 execute on 
a different computer than database server application 148. Furthermore, these servers 
may be geographically separated, for example also coupled over Internet 130. Web 
server application 142 responds to communication it receives from a Web browser 
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112 over Internet 130. Web servo- application 142 listens for connections on a 
particular Internet Protocol (IP) port and receives standard http (hyper-text transport 
protocol) requests on that port that identify particular Web pages that the requestor, 
typically Web browser 112 on a client computer 110, wants to receive. Web pages 
5 144 includes individual Web pages which are stored as individual files on a file 

system. Web server application 142 responds to http requests from Web browser 112 
by providing HTML (hyper-text markup language) format data to the Web browser, 
which in turn uses the HTML data to display information to user 120. Requests from 
a Web browser 112 can include data, for example obtained by user 120 filling out a 

10 form or selecting options by selecting choice buttons with a mouse. 

Web server application 142 optionally communicates with a commerce 
application 156 that provides an interface for Web server application 142 to external 
commercial transaction and order fulfillment services 158, which allow user 120 to 
order products that have been recommended by the system. 

1 5 Server computer 140 also includes a database maintenance application 152. 

An administrator 154 of the system uses the maintenance application to update the 
data stored in database 150. For example, as new products are added to the inventory, 
administrator 154 enters information related to the new product and this information 
is stored in database 150. 

20 Referring to FIG. 2, Web pages 144 includes two types of pages. One type is 

static Web pages 210, which are HTML format pages that are passed by Web server 
application 142 (FIG. 1) directly to a requesting Web browser 112 without 
modification. The other type is active server pages 220. An active server page 220 
includes a procedure specification that, when requested by a Web browser, that page 

25 is executed under the control of Web server application 142 rather than being passed 
directly to the Web browser. Execution of the procedure specified by an active server 
page 220 produces HTML format data that is passed by Web server application 142 to 
the Web browser. In this embodiment, active server pages 220 include Visual Basic 
language procedure specification that are executed under to the control of Web server 

30 application 142. 
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Active server pages 220 can include references to services provided by 
database server application 148 (FIG. 1). For instance, a Visual Basic procedure in 
active server page 220 accesses database server application 148 through an 
application program interface (API) for the database server application. During 
execution of the stored procedure, Web server application 142 can therefore access 
data stored in database 150. Active server pages 220 can also include references to 
stored database procedures 230. Each stored database procedure 230 includes one or 
more SQL statements. Web server application 142 invokes a stored database 
procedure 230 during execution of an active server page 220, Database server 
application 148 controls the execution of the stored database procedure 230 to provide 
return data to Web server application 142. 

Together, static Web pages 210, active server pages 220, and stored database 
procedures 230 determine the user interface through which user 120 interacts with the 
system. 

Database 150 includes a number of separate tables. A product table 240 
includes information related to an inventory of products available in the system. A 
skill hierarchy table 244 relates to a set of skills, or learning goals. Both product table 
240 and skill hierarchy table are dynamic in that they can be modified, for example, 
as new products are added or as characterizations of existing products are refined. 
This set of skills is generic, not being particularly related to any one of the products. 
The set of skills represented in skill hierarchy table 244 is used to represent attributes 
or characteristics of the products represented in product table 240. The set of skills is 
also used in part to represent attributes that are desired to be associated with 
recommended products for child 122. A third table, skill/product table 242 is used to 
associate particular products with particular skills in a many-to-many mapping 
between products and skills. Product table 240, skill hierarchy table 244, and 
skill/product table 242 together provide data needed to recommend particular products 
based their relationships to a desired skill or set of skills. 

Database 150 also includes permanent user data 248, which include 
information about users 120 and children 122 that is persistent between sessions in 
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which the user interacts with the server computer. Information about a user 120 
includes their first and last name, and a password that is needed to access the system. 
Information about a child 122 includes their name, their date of birth, and their current 
grade. 

Database 150 also includes a user cache 246, which is used to store 
intermediate results while the system interacts with a user 120 during a session. User 
cache 246 is used to avoid recomputing results during a particular session. 
Information in user cache 246 has a limited lifetime. For example, data in user cache 
246 is periodically removed if it has not been accessed for a predetermined amount of 
time. 

Referring to FIGS. 3, database 150 (FIGS. 1, 2) includes tables that relate 
skills and products. Product table 240, skill/product table 242 and skill hierarchy 
table 244 are interrelated and include entries that refer to entries in other of these 
tables. Product table 240 includes one product record 330 per product in the 
inventory of the system. Each product record 330 includes a number fields related to 
the product represented by that record. A product identification 312 includes a unique 
identifier that is used to refer to this product in other tables of the database. A 
description 314 includes fields that described the product including, for example, a 
text description, a universal product code, and a manufacture or supplier. 

Each product record also includes a learning styles field 316. Learning styles 
316 includes a number of Boolean indicators used to signify whether or not that 
product is related to one of a corresponding number of standard learning styles. In 
this embodiment, seven learning styles are used. The specific learning styles are 
motivated Howard Gardner's theory of Multiple Intelligences. These learning styles 
are: 
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1. Physical, 

2. Interpersonal, 

3. Intrapersonal, 

4. Linguistic, 

5. Mathematical, 

6. Musical, and 

7. Visual. 

Each product record 330 also includes a grade range 318, which includes a 
lower and upper grade for which the product is generally appropriate. Note that more 
refined grade ranges are associate with particular skills related to the product, as is 
described below with reference to skill/product table 242. 

Each product record 330 also includes a learning approach field 320. 
Learning approach 320 includes a number of Boolean fields, each indicating whether 
the product is related to a corresponding learning approach. These learning 
approaches are: 

1. Instruction, 

2. Reference, 

3. Drill & Practice, 

4. Exploration & Discovery, 

5. Tools, and 

6. Educational Game. 

Finally, each product record 330 includes a number of ratings 322 related to a 
corresponding number of ratings categories. Each rating field includes a numeric 
rating, from 1 (worst) to 5 (best), for each of 5 ratings categories. These ratings 
categories are: 

1. Educational Value, 

2. Fun, 

3. Ease of Use, 

4. Depth/Reusability, and 

5. Reviewer's Opinion. 
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Skill hierarchy table 244 defines a hierarchical relationship of a set of skills. 
A skill hierarchy 380 is illustrated in a portion of a tabular form. A complete listing 
of skill hierarchy 380 used in this embodiment is shown in FIGS. 4A-D. Each skill is 
characterized by three identifiers: a major skill 382, a minor skill 384, and a detailed 
skill 386. For example, as shown in the figure, the "reading & writing" major skill 
includes the "general," "phonics," and "writing" minor skills. Each combination of a 
major and minor skill includes one or more detailed skills. For example, the "reading 
& writing/phonics" major/minor skill includes the detailed skills "beginning sounds" 
and "ending sounds." As is discussed further below, during the matching process, the 
relationship of different detailed skills according to whether they fall under the same 
major/minor skill combination, the same major skill, or under different major skills 
can affect the selection of products. 

Skill hierarchy table 244 includes the interrelationship of the detailed skills, 
and associates each detailed skill with a skill identifier 362. Therefore, each skill 
identifier 362 is associated with a particular combination of a major skill 364, a minor 
skill 366, and a detailed skill 368. Note that in general, each major skill includes a 
"general" minor skill, and each combination of a major skill and a "general" minor 
skill includes a "general" detailed skill. Therefore, a major skill as a whole is referred 
to by the identifier for the "general/general" minor/detailed skill combination for hat 
major skill. Similarly, a minor skill is referred to by the identifier of the "general" 
detailed skill for that minor skill. 

Skill/product table 242 provides a link between product records 330 in product 
table 240 and skills hierarchy 380. As a whole, skill/product table 242 represents a 
many-to-many skill/product mapping 390 between products and skills in skills 
hierarchy 380. Each record of skill product table 242 is related to an association 
between a single of the products and a single of the skills. A representative skill 
product record 352 relates a product with product identification 113 with a skill with 
skill identification 25. Each skill/product record includes a skill identification field 
342 and a product identification field 344. Each record also includes a grade range 
field 346. Unlike grade range field 318 in product table 318, this grade range field 



SUBSTITUTE SHEET (RULE 26) 



WO 00/30010 



PCT/US99/27140 



-12- 

relates to both the product identified in the product identification field 344 and the 
skill identified in skill identification 342. A particular product may have educational 
value for one skill in a first grade range, while it has educational value for another 
skill in a second grade range. The values in grade range 346 in skill/product table 242 
provide a mechanism for representing such skill-dependent grade ranges. Each 
skill/product record also has a rating field 348. A particular product may have strong 
educational value for one skill, a moderate value for a second skill, and a minor value 
for a third skill. Rating 348 encodes the strength of the educational value using a 
three level encoding, using the numbers 1 to 3, with 3 being the greatest strength and 
1 being a lesser strength. Skill product records 352, 350, and 354 are all related to a 
product will product identification 1 13 and skills 25, 12, and 28, respectively, and 
strengths of the relationships, 3, 2, and 1, respectively. 

In this embodiment, a reviewer assigns the strength of these relationships 
according to the following definitions: 

3. Product addresses the skill explicitly and exclusively. If the product 
addresses more than one skill and the treatment of this skill compares 
favorably with products that address this skill exclusively, the reviewer 
could also assign this score, however this should be done with 
reservation. 

2. Product addresses the skill explicitly, but as part of a general treatment 
of the subject area. An example would be a workbook that addressed 
spelling, phonics, vocabulary, and usage. Each of these skills would 
be rated 2 for this product. 

1 . Product addresses the skill as a secondary subject. The product does 
not explicitly address the skill. An example would be a science project 
book that utilizes math computation and reading comprehension skills. 
Math computation and reading comprehension for this product would 
be rated as 1 

Referring to FIGS. 4A-D, a snapshot of a typical skills hierarchy 380 includes 
161 different skills, arranged in seven major skills: Creative Arts, Developmental, 
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Logical Reasoning, Math, Reading & Writing, Science, and Social Studies. It should 
be recognized that the skill hierarchy may be extended as new products are added to 
the inventory, and that skills may be added to the skill hierarchy to characterize new 
or existing products. For example, detailed skills for "Math/Algebra" (FIG. 4B), 
5 which here only includes the "General" detailed skill, could be added by an 
administrator 154 (FIG. 1) if a new products being added to the inventory were 
particularly related to a particular aspect of algebra. Alternative embodiments of the 
system may use fewer or a greater number of skills, which may be arranges under a 
different number of major skill categories. Alternative embodiments of the system 

10 may also use a different hierarchical structure to arrange skills. For example, a 
structure may be used which allows an arbitrary number of hierarchical levels. 
Additionally the number of skills in each category, and the number of categories in 
each hierarchical level, may be different. 

Referring to FIG. 5, skill/product mapping 390 illustrates the nature of the 

1 5 association of skills and products. Each association, which corresponds to a single 
record in skill/product table 242 (see FIG. 3), is illustrated as a single link between 
one of the skills illustrated as points on the left and one of the products illustrated as 
points on the right. Links 392, 394, and 396 correspond to skill/product records 350, 
352, and 354, respectively, in FIG. 3. Only a small number of links are illustrated 

20 between the products and the skills. In practice, each product is linked at least one, 
and typically, multiple skills, and each skill is linked to multiple products. 

Referring to FIGS. 6A-C, information related to a child 122 is obtained from a 
user 120 or in some operating modes, is obtained directly from the child. Referring to 
FIG. 6A, a user 120 provides an age 620 and a grade 622 for the child. The user 

25 provides this by filling out a form that is displayed by Web browser 122 on client 
computer 110 based on HTML data provided by Web server application 142 to Web 
browser 112 (see FIG. 1). 

In operation of the system, user 120 accesses the system controlling Web 
browser 1 12 to request a particular Web page from server computer 140. This begins 

30 a interaction between user 120 and the system that follows the general model of many 
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Web based systems. The interaction alternates between the system providing HTML 
data to Web browser 1 12, which the Web browser uses to form a display to the user, 
and the user selecting a next Web page or providing data that results in a next request 
being sent by Web browser 112 to server computer 140. 

Several operating modes are supported by the system. In general, in each 
operating mode, user 120 provides information about child 122 to the system, and the 
system in return recommends products to the user based on that information. The 
operating modes differ in the nature of the information provided by the user, and in 
the particular matching algorithm used by the system to choose products to 
recommend. In the discussion below, the different modes of providing information to 
the system are first described, followed by a description of the corresponding 
matching algorithms. 

In a first operating mode, user 120 selects a primary learning style 626, and 
optionally selects a secondary learning style 628 from a list of the seven learning 
styles identified above that is displayed to the user by the system. The system then 
uses these learning styles to match the products in the inventory. 

In a second operating mode, rather than directly selecting primary learning 
style 626 and secondary learning style 626, user 120 interacts with a learning style 
survey 612. Learning style survey 612 is implemented by having Web browser 1 12 
display a series of questions to user 120. The questions are obtained from Web server 
application 142. The user's responses to the questions are passed back to Web server 
application 142, where the primary and secondary learning styles are determined from 
the responses. Three sets of 28 learning style questions 610 are stored on server 
computer 140. Each set is associated with a different grade range. The system selects 
the appropriate set based on grade 622 that was provided to the system by user 120. 
The system presents the questions in the selected set to the user. A sample set of 28 
questions is shown in FIG. 7. For each of the enumerated statements in FIG. 7, user 
120 is asked to respond with one of three choices related to how well the statement 
matches child 122 ("rarely," "sometimes," "usually"). Each question is associated 
with one of the seven learning styles, yielding four questions per style. A response of 
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"usually" adds 3 to a tally for the corresponding style, "sometimes" 2, and "rarely" 1. 
The learning style with the highest tally is selected by the system to be primary 
learning style 626, and the second highest tally is selected to be secondary learning 
style 628. 

In a third operating mode, user 120 enters a learning goal 624 for the child by 
selecting from a list of skills in skill hierarchy 850 (FIGS. 4A-D). In one alternative 
approach to selecting the learning goal, user 120 first selects a major skill from a first 
"pull-down" list. Having selected the major skill, the system provides a list of minor 
skills for that major skill for the user to select from. Finally, the system provides a list 
of detailed skills that the user can select from. The system then uses the selected 
learning goal 624 to match to and sort the products in the inventory. This matching 
procedure is described in detail below. 

In a fourth, combined, operating mode, the system makes use of learning goal 
624, which is provided directly by user 120, as well as primary learning style 626 and 
secondary learning style 628, which are either provided directly by user 120 as in the 
first operating mode, or are produced by learning style survey 612, as in the second 
operating mode. 

Referring to FIG. 6B, in a fifth operating mode, rather than user 120 directly 
providing learning goal 624 to the system, the system determines a skill assessment 
640, which includes one or more skills 622 and a corresponding grade equivalent 644 
for each of those skills. In this operating mode, called an "on-line skills checkup," 
child 122 responds to a set of skill-dependent questions 630 that are presented to the 
child by a skills test 632. Skills test 632 is implemented as a procedure in which Web 
browser 112 presents skill-dependent questions 632 to child 122, who responds to the 
questions on computer 1 10. In alternative embodiments, the questions are presented 
to the child "off-line," for example printed on paper, and user 120 enters the answers. 
Skills test 632 makes use of score norms 634 associated with the skills tests. Score 
norms 634 provide the data needed to convert scores on the skills test into grade 
equivalent 644. 
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One example of a skills test 632 is illustrated in FIGS. 8A-E. This test is 
related to the "ReadingftWritingTReading^Comprehension" 
(major/minor/detailed) skill, and is intended for children in grades 4-6. A child is 
presented one or more of the five passages shown in FIGS. 8A-E in order, and 
answers a set of six questions about each passage. The passages are intended to 
correspond to grade levels 3 through 7. If a child gets five out of the six questions for 
a grade level correct, the child has demonstrated that grade level and moves to the 
next story. In this example, skills assessment 640 includes grade equivalent 644 that 
is determined from the score norms 634 for this test, that is, that a child obtains at 
least five out of the six questions correct on the highest grade level passage. 

Other skills tests 634 address multiple skills. For example, various detail 
skills under the "Math"/"Decimal" major/minor skill category (e.g., "Addition," 
"Subtraction," "Division") are addressed by one skill test 634. Each question is 
associated with a one of the detail skills. Using score norms 634 for those questions, 
a different grade equivalent 644 may be determined for each detail skill. 

A sixth operating mode uses a similar approach to the on-line skills checkup. 
However, rather than having questions presented by Web browser 112, an off-line 
testing application 116 (FIG. 1) is executed on computer 110. Testing application is 
similar to an on-line skills checkup, however, a wider range of skills is typically 
addressed and more questions are asked of the child. Execution of testing application 
116 may perform the entire skills test 632 (FIG. 6B) producing skill assessment 640, 
or alternatively, part of skills test 632 may be performed at server computer 140. For 
example, testing application 116 may produce raw test scores, and a procedure on 
server computer 140 would then application score norms 634 to determine the grade 
equivalents for the tested skills. Rather than entering answers or test scores directly to 
Web browser 112, the system has the provision to transfer data stored by testing 
application 116 to server computer 140, for example, under the control of user 120 
through a Web browser interface, or directly from testing application 116. 

Referring to FIG. 6C, a seventh operating mode is related to the on-line and 
off-line modes illustrated in FIG. 6B. In this mode, child 122 has taken a 
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standardized test 652, such as the Michigan Educational Assessment Program 
(MEAP) Information test or the Massachusetts Comprehensive Assessment System 
test. These tests include a set of standardized test questions 650. Typically, 
government sponsored agency scores the tests and produces standardized test scores 
654 for each child, broken down into a number of subject areas. 

In this operating mode, user 120 enters standardized test scores 654 into the 
system through the Web browser interface. The system performs a skill mapping 656 
to convert the scores into a skill assessment 660, of the type produced by the on-line 
test Skill mapping 656 performs two functions. First, each subject area for which a 
standardized score is obtained typically maps into multiple skills in the skill 
hierarchy. Second, for each of the subject scores the system determines a grade 
equivalent. In normed tests, the system derives the grade equivalent based on 
statistical data associated with the test and which characterizes the performance of 
individuals at various grade levels that have previously taken the test. In criterion 
tests, the system derives the equivalent grade level based on data which characterizes 
at which grade level various skills tested should be mastered. The result is that a skill 
assessment 660 typically includes more skills 662 than the number of subject scores 
in standardized test scores 654, and for each skill, skill mapping 656 has assigned a 
grade equivalent 664. 

In some embodiments, the function of obtaining standardized test scores 654 
and transferring them to server computer 140 is performed by a third party using third 
party computer 160. For example, the third party may have the ability to obtain the 
standardized test scores directly from a government agency. 

Other than in the first and second operating modes, the system matches 
products based at least in part on a skill associated with child 122. Referring back to 
FIG. 6A, in the first and second operating mode, user 120 has either directly specified 
primary learning style 626 and possibly secondary learning style 628, or these 
learning styles were determined using learning style survey 612. User 120 has also 
provided grade 622 for the child. 
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Based on the learning styles and the child's grade, Web server application 142 
(FIG. 1) initiates a stored procedure 230 (FIG. 2) that performs a query and sort of the 
products that match the child's grade and learning styles. All products for which the 
child's grade is in the product's grade range 318 (FIG. 3), and for which either the 
primary or the secondary learning styles are indicated in the product's learning 
approach 316 (FIG. 3) are matched. The products are sorted according to a score that 
is 1 if only the secondary learning style matched, 2 if only the primary learning style 
matched, or 3 of both the primary and secondary learning styles matched. Within 
each of these scores, the products are further sorted by an average of the product's 
ratings 322 (FIG. 3). The highest scoring products are then presented to the user, for 
example, on a series of screens. In some embodiments, particular learning approaches 
320 (FIG. 3), such as "Drill&Practice" are prioritized higher within products that 
match the learning styles equally. 

In the remaining operating modes, a selected learning goal 624, which is a 
skill from the skill hierarchy, or skills in a skills assessment, are used in the matching 
process. Referring to FIG. 9, the approach to associating skills with products is 
illustrated. At the left of the figure, a list of desired skills 910 includes two skills 912 
that are to be matched to the inventory of products 970, shown on the right. The 
mapping of a particular skill 912 to one or more products 970 makes use of skills 
hierarchy 380 and skill/product mapping 390, which are store in skill hierarchy table 
244 (FIG. 2) and skill/product table 242 in database 150. 

First, a skill 912 is mapped to all skills within the same major skill 382 in skill 
hierarchy 380. The mapping is weighted so that matching skill has a higher weight, 
than skills that are within the same minor skill, which have a higher weight than skills 
that are only within the same major skill. In this embodiment, these weights are 3, 2, 
and 1, respectively. Therefore, skill hierarchy 380 provides a basis for expanding a 
small number of targeted skills 912, into a larger number of skills, each weighted by 
its relationship to a target skill. 

Second, product skill mapping 390 is used to associate particular products 
with the expanded set of skills. Recall that each association 956, which corresponds 
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to one record in skill/product mapping table 242 (FIG. 3) is also associated with a 
rating, which is a number in the range 1 to 3 with 3 being the strongest association 
and 1 being a weaker association. In one embodiment, a score for a product is the 
highest product of the weighting of an expanded skill and the weighting of an 
5 association linking that expanded skill and the product Optionally, the relative 
contributions of the weighting of an expanded skill and that of a rating of an 
association can be adjusted by multiplying one or the other by a fixed constant. If an 
3-rated association links a target skill 912 and a product, that product receives a high 
score. If only a 1 -rated association 956 links a skill within the same major skill as the 

1 0 target skill and a product, that product receives a relatively low score. 

Alternative embodiments combine the effects of different target skills 912, or 
different expanded skills in different ways. In one alternative, the best match is used 
to score a product. In another alterative, a sum of scores related to a product is used. 
That is, if a product is associated with multiple of the target skills, its score is the sum 

15 of the scores associated with each of the target skills, rather than the maximum of 
those scores. In another embodiment, matching is equivalent to matching each of the 
target skills separately, and then combining the sorted lists, thereby ensuring that at 
least some products related to each of the target skills are at the top of the list of 
recommended products. 

20 In operating modes in which a grade equivalent is associated with a target 

skill, as in the case in which a skill assessment for one or more skills has been 
generated, only associations with a grade range including the grade equivalent for a 
skill are used. In some alternative embodiments, user 120 is asked by the system 
whether products that emphasize the child's strength or products that emphasize the 

25 child's weaknesses are desired. If the user respond that emphasizing the child's 
strength is desired, only the skills associated with the highest determined grade 
equivalents are used, or alternatively, the skills for which the grade equivalent is 
greater than the actual grade entered for the child. Similarly emphasizing weaknesses 
involves using skills with relatively low grade equivalents. 
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In some embodiments, yet other numerical scores are combined into a total 
score for each product For instance, score related to one or more of the product's 
ratings may be combined, thereby including more products with high ratings at the top 
of the recommended list. Also, a "product lifetime" related to the number of grades 
left between a child's grade level and a product's maximum recommended grade is 
factored in, thereby including products that should be relevant to the child for more 
years higher in the list of recommended products. Another alternative score to 
combine into a product's overall score is a "focus" score that is higher for a product 
that is associated with a smaller number of skills than for a product that is associated 
with a large number of skills. 

In some embodiments, the numerical score obtained for a product based on the 
matching to one or more target skills is combined with a score determined from a 
child's primary and secondary learning styles. 

In alternative embodiments, alternative structures than skills hierarchy 380 are 
used to represent the interrelationship of skills or other attributes used to characterize 
the products. For example, pair-wise relationships between skills may be explicitly 
stored, or skills may belong to overlapping categories. In each of these alternatives, 
specification of one desired skill allows the system to derive a set of related skills that 
may be matched with products in the product inventory. 

Referring back to FIG. 1, administrator 154 uses database maintenance 
application 152 to maintain database 150. When a new product is added to the 
inventory of products, a new product record 330 (FIG. 3) is added to product table 
240. In addition, one or more skill/product records are added to skill/product table 
242, each record associating the new product with one of the existing skills in the skill 
hierarchy. If the new product is related to a skill not yet in the skill hierarchy, that 
new skill can first be added by adding a record in skill hierarchy table 244. Then a 
skill/product record associating the new product to the new skill is added to 
skill/product table 242. In this way, the set of skills is extensible without affecting 
existing data that associates previously entered products with previously defined 
skills. 
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Refening back to FIG. 1, in alternative embodiments, other forms of data 
networks couple client computers 110 and server computer 140. For example, a 
private "intranet" can couple the client and server computers. Also, rather than 
coupling the computers over a data network, client computers 110 can alternatively 
connect directly to server computer 140, for instance using dialup telephone modem 
connections, or dedicated connections such a leased telephone lines. 

The system described above makes use of Web browser 112 executing at 
client computer 110 interacting with Web server application 142 executing at server 
computer 140. Alternative types of applications and servers can be used. For 
example, a client application can execute at client computer 110 and communicate 
with an server application executing at server computer 140 using a proprietary 
protocol 

Furthermore, the system shown in FIG. 1 makes use of a client/server 
architecture in with a particular division of functions between the client and server 
computers. In alternative embodiments, different divisions of functions are used, and 
different types of databases and database procedures may be used. For instance, 
server computer 140 may host database 150, but the software that performs the 
functions of the database procedures, such as Java applets, can be transferred to client 
computer 110 and executed there to access database 150 remotely over Internet 130. 

In FIG. 1, server computer 140 is shown as a single computer. In alternative 
embodiments, the functions of server computer 140 can be distributed among several 
computers. For instance, database server application 148, database procedures 146 
and database 150 may be hosted on a different computer than Web server application 
142 and Web pages 144. Furthermore, multiple Web server applications 142 and 
Web pages 144 may be hosted on different computers, and interconnected by links in 
the Web pages returned to the client computers. 

It is to be understood that the foregoing description is intended to illustrate and 
not to limit the scope of the invention, which is defined by the scope of the appended 
claims. Other embodiments are within the scope of the following claims. 
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WHAT IS CLAIMED IS: 

1 . A computer-implemented educational product recommendation system 
comprising: 

a storage for data characterizing relationships between product attributes 
5 within a set product attributes, wherein the set of product attributes includes a set of 
educational skills; 

a storage for data associating each of a plurality of products with one or more 
of the product attributes; and 

software stored on a computer-readable medium for causing a computer to 
1 0 determine one or more desired product attributes, 

access attribute relationship data from the storage for data 
characterizing relations between the product attributes, 

determine related product attributes based on the desired product 
attributes and the attribute relationship data, 
1 5 access product association data from the storage for data associating 

products with product attributes, and 

determine a matching set of the products using the product association 
data and the desired and related product attributes. 

2. A system of claim 1 wherein the storage for associating each of the 
20 plurality of products with one or more of the product attributes includes a database 

table for holding a plurality of records, each record associating one of the products 
with one of the product attributes. 

3. The system of claim 1 wherein the relationships between product 
attributes include inclusion of particular ones of the educational skills within the same 

25 category of educational skills, and wherein determining related product attributes 
includes determining educational skills that are in the same category of educational 
skills as a desired educational skill. 
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4. The system of claim 3 wherein the relationships between product 
attributes includes inclusion of a number of categories of educational skills within a 
broader category of educational skills. 

5. The system of claim 3 wherein determining one or more desired 
product attributes includes determining test scores for an individual, and mapping the 
tests scores to a quantity related to one of the educational skills. 

6. The system of claim 5 wherein the quantity related to one of the 
educational skills is an equivalent grade level. 

7. Software stored on a computer-readable medium for causing a 
computer system to: 

determine one or more desired product attributes from a set of product 
attributes, wherein the set of product attributes includes a set of educational skills, 

access attribute relationship data from a storage for data characterizing 
relationships between product attributes in the set of product attributes, 

determine related product attributes based on the desired product attributes and 
the attribute relationship data, 

access product association data from a storage for data associating each of a 
plurality of products with one or more of the product attributes, and 

determine a matching set of the products using the product association data 
and the desired and related product attributes. 
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8. A method for recommending educational products comprising: 
determining one or more desired product attributes from a set of product 

attributes, including determining one or more desired educational skills; 

accessing attribute relationship data from a storage for data characterizing 
relationships between product attributes in a set of product attributes; 

determining related product attributes based on the desired product attributes 
and the attribute relationship data; 

accessing product association data from a storage for data associating each of 
a plurality of products with one or more of the product attributes; and 

determining a matching set of the products using the product association data 
and the desired and related product attributes. 

9. The method of claim 8 wherein determining one or more desired 
educational skills includes: 

providing a questionnaire including a plurality of questions; and 
accepting responses to the questions. 

1 0. The method of claim 9 further comprising: 

determining an equivalent grade level associated with each of the one or more 
educational skills based on the accepted responses. 

1 1 . The method of claim 8 wherein determining one or more desired 
educational skills includes: 

accepting test scored obtained on a standardized test; and 
determining an equivalent grade level associated with each of the one or more 
educational skills based on the accepted test scores. 

12. The method of claim 1 1 wherein determining an equivalent grade level 
includes comparing the accepted test scores with statistical data characterizing a 
distribution of test scores for a grade level. 
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1 3. The method of claim 8 wherein determining related product attributes 
based on the desired product attributes and the attribute relationship data includes 
determining related educational skills that are in a same category of educational skills 
as at least some of the desired educations skills. 

5 14. The method of claim 1 3 wherein determining one or more desired 

product attributes includes determining test scores for an individual, and mapping the 
test scores to a quantity related to one of the educational skills. 
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1 . My child would prefer listening to a story, writing, and/or reading over 
watching television. 

2. My child does mental math with ease. 

3. My child is highly self-motivated. 

4. My child sings/hums even when there isn't any music playing. 

5. My child enjoys drawing, painting and/or sculpting. 

6. My child enjoys being involved in playgroups, clubs, and/or extra-curricular 
programs. 

7. My child enjoys participating in sports or physical activity, usually on a 
regular basis. 

8. My child loves being read to and/or independent reading. 

9. My child enjoys sorting buttons or seashells- by size, shape, color, etc. 

1 0. My child likes being alone to play a video game or solve a puzzle. 

11. My child brings my attention to sounds he/she hears in the environment 
(birds, cars, machinery, etc.). 

12. My child enjoys putting puzzles together. 

13. My child establishes and maintains friendships with ease. 

14. My child finds it difficult to sit still for long periods of time. 

15. My child likes to tell stories, to report about his/her day, and to retell 
events from television. 

16. My child enjoys mixing ingredients together for real or imaginary cooking. 

17. My child is competitive. 

18. My child can mimic musical and rhythmic patterns. 

19. My child is fascinated by maps and graphs. 

20. My child enjoys playing team sports/games. 

21 . My child uses his/her body when communicating (talks with hands, nods 
head, rocks from side to side, etc.). 

22. My child has a good memory for poems, rhymes and/or stories. 

23. My child performs tasks (helps to set the table, cleans up toys) in an 
orderly step-by-step process. 

24. My child is extremely independent. 

25. My child enjoys creating sounds and music with real or homemade 
instruments. 

26. My child has a vivid and active imagination. 

27. My child demonstrates care and concern for friends who need help, are 
injured, or are upset. 

28. My child enjoys building and fixing things. FIG. 7 
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Comprehend passage at the 3rd grade level 

THIS STORY IS ABOUT GIANT TREES. READ THE STORY TO LEARN ABOUT THEM. 

The biggest trees in the world are giant redwoods. They are called sequoias. Many of the 
trees stand two hundred feet high. Some of their trunks are more than twenty-five feet 
through the middle. They may be the oldest living things on earth. Some sequoias are so 
large that one of them would give enough wood for five houses. 



1. What is the best name for this story? 

a. Forest Giants 

b. The Last Trees on Earth 

c. Old Things 

d. How We Get Lumber for Houses 

2. In this story trunk means . 

a. root 

b. main stem 

c. snout 

d. case 

3. How many homes will a sequoia make? 



FIG. 8A 



4. How tall do some redwoods grow? 

a. 25 feet 

b. 5 feet 

c. 200 feet 

d. 100 feet 

5. People believe the redwoods are the oldest living things on earth because 

a. sequoia is an old Indian word 

b. all trees live a long time 

c. there are lots of houses made from them 

d. they are so large 

6. The giant redwoods are found mainly in 

a. Canada 

b. South America 

c. California 

d. Wisconsin 



a. 25 

b. 5 

c. 200 

d. 1 
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Comprehend passage at the 4th grade level 

THIS STORY IS ABOUT A BOY AND HIS HORSE. READ THE STORY TO FIND OUT HOW HE GOT 
HIS HORSE. 



Colonel Carter presented me with a colt. I had my pony, and my father meanwhile had 
bought a pair of black carriage horse and a heifer. I had to attend to ail of them when we 
had no help. And servants were hard to get and keep in those days. The women married, 
and the men soon quit service to seize opportunities always opening. My hands were 
pretty full, and so was the stable. But Colonel Carter seemed to think that he had 
promised me a colt. He had not; I would have known it if he had. No matter, he thought 
he had, and maybe he did promise himself to give me one. That was enough. The kind of 
man that led immigrant trains across the continent and delivered them safe, sound, and 
together where he promised would keep his word. One day he drove over from Stockton, 
leading a two-year-old that he brought to our front door and turned over to me as mine. 
Such a horse! 



7. The best title for this paragraph is 



a. Colonel Carter's Promise 

b. A Gift from the Colonel 

c. The Immigrant Trains 

d. A Stable of Horses 

8. In this paragraph attend means . 

a. heed 

b. be present 

c. escort 

d. look after 



9. The author regarded Colonel Carter as the kind of man who would 

a. take critical measures 

b. make good his word 

c. engage in dangerous undertakings 

d. give away his last possession 

10. Before receiving the colt, the author was responsible for 

a. two animals 

b. three animals 

c. four animals 

d. five animals 



1 1 . Colonel Carter was probably 



a. an old friend of the family 

b. an old forgetful man 

c. a stranger 

d. a horse trader 

12. Servants were also probably hard to get because . 



a. the work was disagreeable 

b. the frontier was so thinly populated 

c. there were moral objections to serve C I fl OD 

d. the wages were too low llu. OD 
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Comprehend passage at the 5th grade level 

THIS STORY IS ABOUT HOW NATIVE AMERICANS IN THE EARLY PART OF THE CENTURY 
COOKED A SPECIAL FOOD. READ THE STORY TO FIND OUT WHAT THE FOOD IS AND HOW THE 
FOOD WAS COOKED. 

In the early part of the century, Native Americans discovered several ways to pop corn. 
They might roast the ear of com over an open fire. Then they would gather up the 
popcorn that popped free of the flames. Sometimes they shelled the com and then threw 
the kernels directly into the fire. Other times they stirred the kernels into a shallow day 
bowl filled with sand. When the sand reached a high temperature, the com would pop out 
of the sand. 



13. The best name for this story is . 

a. The History of Popcorn 

b. Why Corn Pops 

c. Three Methods of Preparing Popcorn 

d. How Native Americans Served Popcorn 

14. In this story, free means . 

a. not busy 

b. without charge 

c. having liberty 

d. loose 

15. Native Americans sometimes made popcorn by . 

a. freezing the com first 

b. cooking the whole ear of corn 

c. dropping com into boiling water 

d. leaving the com in the sun 

16 The sand in the clay bowl 

* 

a. heated the corn 

b. dried out the com 

c. made the popcorn tasty 

d. made the popcorn weigh more 

17. Native Americans always made popcorn with 

a. pots 

b. steam 

c. clay bricks 

d. heat 

1 8. Popcorn is made from . 

a. a special legume 

b. all kinds of com 

c. a certain kind of corn 

d. cornflakes 

FIG. 8C 
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Comprehend passage at the 6th grade level 



THIS IS A STORY ABOUT A COMMON FARM TOOL READ TO FIND OUT HOW IT HAS BEEN USED 
OVER THE YEARS. 

The plow, which can be made of wood or steel, is a wedge that is dragged through the 
soil by an animal or a tractor. It slices away the top layer of soil and overturns it. In this 
way, furrows are created for planting crops. In addition, vegetation on the surface of the 
soil is buried so that it rots and provides fertilizer for the new crops. The plow is an 
ancient piece of agricultural equipment. Wooden plows have been employed for several 
thousand years. Metal plows date back about two centuries. 

1 9. The best title for this paragraph is . 



a. Soil Science 

b. Principles of Farming 

c. The History of Agriculture 

d. A Basic Farming Tool 



20. In this paragraph, employed means . 



a. practiced 

b. hired 

c. operated 

d. manufactured 

21 . The main purpose of a plow is to 



a. sow seeds for crops 

b. mix fertilizer into the soil 

c. form trenches in the soil 

d. harvest ripe crops 



22. The earliest plows were . 



a. made of wood 

b. made of steel 

c. made of iron 

d. pulled by tractors 

23. Much of the vegetation buried by the plow is probably _ 



a. seedlings 

b. small bushes 

c. the old crop 

d. the new crop 



24. In ancient times plows were m 



a. pulled by tractors F I fn ft D 

b. pulled by oxen 1 ,VJ- UL/ 

c. made in factories 

d. powered by wind 
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Comprehend passage at the 7th grade level 

ANIMUS STORY * B ° UT BEARS * REA ° T ° RND 0UT M0RE A 801 ^ THESE INTERESTING 

Many different sorts of bears live in North America. The polar bear is found near the Arctic Circle and 
gives birth to its young only in the winter. The North American black bear is found east of the Rocky 
Mountains and has small paws suited for scratching in the earth as well as climbing trees. The grizzly a 
kind of brown bear, usually lives west of the Rock and is the fiercest of all American bears. 

25. The best name for this story is ' 



a. Alaskan Bears 

b. What Bears Eat 

c. Types of Bears 

d. Bears of the Rockies 



26. In this story suited means . 



a. dressed 

b. pleased 

c. fitting 

d. arranged 

27. The most dangerous bears are located 



a. west of the Rocky Mountains 

b. in South America 

c. in the land of ice and snow 

d. east of the Rocky Mountains 



28. Polar bear cubs are bom 



a. close to the water 

b. in the coldest season 

c. in the spring 

d. in forest caves 



29. Black bears probably live on 



30. Polar bears are named for 



a. seals 

b. berries and insects 

c. fish 

d. other bears 

FIG. 8E 



a. a place on the map 

b. a famous explorer 

c. the color of their fur 

d. their lazy habits 
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